Skip to content

fix(runtimes): resolve a runtime selector to the key that matches exactly - #497

Merged
rominf merged 1 commit into
mainfrom
fix-runtime-selector-exact-key
Oct 2, 2026
Merged

rominf merged 1 commit into
mainfrom
fix-runtime-selector-exact-key

Conversation

@rominf

@rominf rominf commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator
  • If this PR fixes a bug, searched tests/e2e-cucumber/expectations.toml for the fixed ticket ID and removed/narrowed any now-stale xfail rows. — no rows reference this area.
  • If this PR adds a new subcommand or subsystem, its domain implementation lives in its own file per docs/architecture.md. — N/A.
  • Every new or changed user-facing message was read against the code path that runs after it, and its test asserts the resulting state — not only the wording, per AGENTS.md §3. — the one new message is an ambiguity error; its test asserts the resolution outcome, and the behavioural tests assert registry state after a real removal.

Fixes #496

Summary

rocm storage remove-old-installs decides which install to remove by inspecting a manifest, then names it to the deleter by key string — and the two sides resolved that key differently. The planner matched exactly; select_runtime_manifest matched case-insensitively and took the first hit from a newest-first list. With two registry keys differing only in letter case, prune decided about one install and acted on the other.

That pair is reachable without anything exotic: generated keys are slugified to lower case, while runtimes adopt --runtime-key takes the key the user typed and runtimes import takes it from the supplied manifest — neither slugifies. Both record the result read-only, and adopt stamps installed_at_unix_ms with the moment it ran, so the adopted record is newest and won the lookup. The registry's duplicate guard compares file paths, and ABC.json and abc.json are two files on a case-sensitive filesystem.

The effect is not a deleted folder — an adopted record is read-only, so its install root is left alone — but the adopted runtime is silently de-registered, the install the user confirmed stays, and the command reclaims nothing while naming two different runtimes in one sentence.

Risk: low-medium. It changes a resolver shared by seven call sites, but behaviour differs only when two installed keys differ solely in case. Every caller was enumerated; those that pass a stored key (rollback, the uninstall revalidation, prune itself, engine env selection) always hit the exact branch.

Why the resolver, and not the deleter

Handing the deleter the manifest the planner already resolved looks like the smaller fix, and it was rejected: several callers resolve the same key more than once and must land on the same record every time. The uninstall revalidation re-derives its plan from disk after a confirmation prompt that can block indefinitely — deliberately, so a concurrent change cannot be applied against stale data — and activate, rollback, update and engine selection all round-trip a key. Fixing the deleter would have fixed prune alone and left the identical mismatch in the revalidation path.

What the new behaviour is

An exact key match is taken first and wins outright. The case-insensitive fallback — a deliberate convenience, so a key typed in the wrong case still resolves — applies only when it lands on a single record. Two case twins are reported as the ambiguity they are, listing both keys, mirroring the existing error for a runtime_id that names several installs.

That error is unreachable for a user whose keys do not differ only in case: two all-lowercase keys that match case-insensitively are byte-equal, and the registry holds one file per key. The user who does see it previously got a silent, order-dependent pick between two different installs — for activate and uninstall, an action on the wrong runtime.

Test plan

Both regression tests fail before this change and pass after; verified by reverting only the resolver body.

  • runtime_selector_prefers_the_exact_key_over_a_case_twin — builds the twin in memory, so it runs on every platform including the Windows lane, and is the one that goes red on revert.
  • prune_removes_the_install_it_named_and_not_a_case_twin_of_it — drives real registry files, real install folders and the real removal call, and additionally asserts the adopted record still resolves as read-only so no folder becomes deletable. Ignored off Linux: two entries differing only in case cannot be staged on a case-insensitive filesystem, where the second fixture would overwrite the first.

cargo test --workspace --all-targets 3260 passed / 0 failed (exit 0); cargo clippy --locked --workspace --all-targets -- -D warnings, cargo fmt --all --check, cargo xtask manifest --check, cargo xtask tpn --check all exit 0.

On the Gherkin requirement

No scenario is added, and that is a gap rather than an exemption. Scenarios do exist for these commands — runtimes adopt (@id:runtime-adopt-preexisting-rejected, runtime-07) and runtimes import, activate, rollback and uninstall (runtime_lifecycle.feature, e.g. runtime-lifecycle-06) — but none covers two installed runtimes whose keys differ only in case, which is what this fix is about. An earlier version of this paragraph said no scenario existed for these commands at all; that was wrong.

The new ambiguity error is also reachable from user-typed selectors, not only internal round-trips: runtimes activate, runtimes uninstall and update --runtime all resolve through select_runtime_manifest, so a user with a case-twin pair in their registry now gets an explicit error there instead of a silent wrong pick. Covering that end to end needs a multi-runtime install fixture; today the behaviour is covered by a unit test and a filesystem-level test driving the real registry and the real removal path.

Follow-ups deliberately not in this PR

  • Rejecting two registry keys that differ only in case at write time. It cannot replace this fix — registries already on disk can hold such a pair — and it adds a new failure mode to adopt/import with its own questions.
  • rocm runtimes uninstall on one twin still clears the other twin's active pointer and deletes its activation marker, because those comparisons are case-insensitive. That loses state for a runtime still installed. Prune cannot reach it, because the retention holds keep both twins.
  • adopt --runtime-key " abc " stores a key no selector can ever resolve, since the selector is trimmed first.

…ctly

`storage remove-old-installs` decides about a manifest and then names it to
the deleter by key -- and the two sides resolved that key differently.
`build_prune_plan` matched `manifest.runtime_key == runtime_key`, while
`select_runtime_manifest` matched `eq_ignore_ascii_case` and took the first
hit from a newest-first list. With two registry keys differing only in
letter case, prune decided about one install and acted on the other.

That pair is reachable without anything exotic: generated keys are
slugified to lower case, while `runtimes adopt --runtime-key` and
`runtimes import` store the key verbatim, without slugifying it. Both
record the result as read-only, and adopt stamps `installed_at_unix_ms`
with the moment it ran, so the adopted record is the newest and won the
case-insensitive lookup. The registry's duplicate guard compares file
paths, and `ABC.json` and `abc.json` are two files on a case-sensitive
filesystem, so nothing rejects the pair.

The effect was not a deleted folder -- an adopted record is read-only, so
its install root is left alone -- but the adopted runtime was silently
de-registered, the install the user confirmed stayed, and the command
reclaimed nothing while naming two different runtimes in one sentence.

Fixed in the resolver rather than by handing the deleter a manifest,
because several callers resolve the same key more than once and must land
on the same record every time: the uninstall revalidation re-derives its
plan from disk after a confirmation prompt that can block indefinitely,
and activate, rollback, update and engine selection all round-trip a key.
Passing a manifest would have fixed prune alone and left the rest.

An exact key match is now taken first and wins outright. The
case-insensitive fallback -- a deliberate convenience, so a key typed in
the wrong case still resolves -- applies only when it lands on a single
record. Two case twins are reported as the ambiguity they are, listing
both keys, the same way a `runtime_id` naming several installs already is.
That error is unreachable for a user whose keys do not differ only in
case: two all-lowercase keys that match case-insensitively are byte-equal,
and the registry holds one file per key.

Both regression tests fail before this change. The filesystem one needs a
case-sensitive filesystem to hold two such registry entries at all, so it
is ignored elsewhere; the in-memory one runs on every platform, including
the Windows lane, and is the one that goes red on revert.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf requested a review from a team as a code owner October 2, 2026 08:45
@rominf
rominf requested a review from jussielo-amd October 2, 2026 08:45

@jussielo-amd jussielo-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Bug fix — select_runtime_manifest now tries an exact runtime_key match before falling back to case-insensitive matching (erroring on case-ambiguity). Fixes a silent-wrong-deletion bug in storage remove-old-installs when two installed runtimes have keys differing only in case.

Narrowly scoped and unusually well-verified — reverting the resolver locally reproduces the exact reported bug, and the fix restores correct behavior. cargo clippy/cargo fmt --check clean. No dead code, stubs, or bugs found.

Non-blocking suggestions

  • The Gherkin-exemption justification covers remove-old-installs/adopt/import, but the new ambiguity error is also newly surfaced via runtimes activate, uninstall, and update --runtime — user-typed selectors, not just internal round-trips. Worth tightening per AGENTS.md §3.
  • The PR description states no Gherkin scenario exists anywhere for the affected commands — scenarios do exist for adopt (runtime_setup.feature:140-145) and import (runtime_lifecycle.feature:42-50), just not covering this specific case-ambiguity behavior. Worth correcting the wording; the substantive point (no existing coverage of this bug) still holds.
  • docs/testing.md/docs/manual-testing.md are untouched — a judgment call given how narrow the edge case is, but AGENTS.md §5 asks for these when observable behavior changes.

Positive signals

  • Verified empirically, not just read: revert-and-rerun reproduced the exact reported bug, then passed cleanly once restored.
  • Added both a fast cross-platform unit test and a filesystem integration test (correctly ignore-gated to Linux with a confirmed-necessary reason).
  • Reuses runtime_keys_text rather than duplicating, and simplifies a previous inline .map().join().
  • PR description is unusually precise about why the fix belongs in the shared resolver (7 call sites), and proactively discloses a known related out-of-scope bug.

Tradeoff (noted, not a concern)

Fix patches resolution (read time) rather than rejecting case-twin keys at creation (write time) — explicitly and reasonably justified in the PR description as avoiding new failure modes in adopt/import while still fixing registries that already hold such a pair.

@rominf
rominf added this pull request to the merge queue Oct 2, 2026
Merged via the queue into main with commit 1a2a975 Oct 2, 2026
@rominf
rominf deleted the fix-runtime-selector-exact-key branch October 2, 2026 15:55
@rominf

rominf commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

The non-blocking items from the approving review, addressed after merge:

sbmthakur pushed a commit to sbmthakur/rocm-cli that referenced this pull request Oct 12, 2026
…#529)

* test(runtimes): cover the case-twin selector refusal end to end

ROCm#497 made a runtime selector that matches two installed keys only by
letter case fail with an error naming both, instead of silently acting
on one. That is user-observable output, so per AGENTS.md section 3 it
gets a scenario, not only the unit and filesystem tests that shipped
with the fix.

runtime-lifecycle-09 plants two runtimes whose keys differ only in case
(Linux-only: they are two registry files, which can only coexist on a
case-sensitive filesystem), activates with a selector matching both
only by case, and asserts the refusal names both keys AND that nothing
became active. Then it follows the advice: naming one key exactly
activates that runtime and not its twin. Mutation-checked: with the
resolver back to silently taking the first match, the scenario fails at
the refusal step.

Also documents the behaviour in docs/testing.md and
docs/manual-testing.md, which ROCm#497's review asked for.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* test(runtimes): prove the case twins are two files before relying on it

runtime-lifecycle-09's premise is that two keys differing only in case
are two registry files, and `@requires-os:linux` was standing in for that.
It is not the same thing: the isolated root follows `$TMPDIR`, which can
be a casefolded or mounted case-insensitive directory on a Linux host.
There the second plant overwrites the first, the selector has a single
case-insensitive match, activation succeeds, and the scenario failed at
the refusal step blaming the resolver for a fixture that never existed.

The Given step now counts the registry entries and fails, naming the
filesystem, when the twins collapsed. Counting is what works: checking
for the lower-case file would pass on such a filesystem, because that
path resolves to the upper-case one. Checked by planting the same file
twice: the scenario now fails at the Given with that message instead of
at the refusal.

Also stop enumerating commands in docs/testing.md. The refusal is a
property of how any runtime selector is resolved, and the list of three
already missed `engines install --runtime-id`; say that every command
taking a runtime selector refuses instead.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* docs(testing): scope the case-twin refusal to the shared resolver

The paragraph said every command taking a runtime selector refuses a
case-only match. Two do not: ComfyUI's installer keeps its own lookup and
takes the first case-insensitive match, and the config set-default-runtime
and set-engine --runtime-id commands store a selector without resolving
it. Say which commands refuse and name the exceptions.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* test(e2e): skip the case-twin scenario where the temp root folds case

The case-twin selector scenario needs two registry files whose names
differ only in letter case. @requires-os:linux does not guarantee that:
the isolated root follows $TMPDIR, which on Linux and WSL2 can sit on a
casefolded directory or a mounted Windows drive. Such a host cannot hold
the premise, and this suite reports that as not-applicable, not failed.

Add a capability probe that writes two case-twin names under the same
temp root the scenarios use and checks they stayed two files, and a
@requires-case-sensitive-fs gate beside the other host gates. Tag the
scenario with it. The entry count in the Given stays as a guard for
runs that bypass the gates via cucumber's -n/--tags, and now says so.
Share one registry_dir helper between planting and counting so the two
cannot drift.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* docs(testing): reflow the case-twin selector paragraph

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* test(e2e): report the OS before the case-folding reason when both gates fail

On native Windows a scenario tagged both @requires-os:linux and
@requires-case-sensitive-fs failed both premises, and the grid reported
the filesystem reason. Check requires_os first so the coarser reason wins.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* test(e2e): route the remaining registry paths through registry_dir

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

* docs(e2e): list the OS gate before the case-folding gate in resolve()

The doc list still gave the two gates in the order the body had before
the OS check was moved ahead of the case-folding one.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>

---------

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
rominf added a commit that referenced this pull request Nov 6, 2026
`rocm storage remove-old-installs` deletes multi-gigabyte install folders
from a set the user never listed, so its safety rules are worth testing
over generated input rather than examples alone.

Property tests for the pure policy (`select_runtimes_to_remove`), 2048
cases each:

- the install in use, the rollback target, anything the active-runtime
  marker names and anything ROCm CLI does not own are never selected,
  including at `--keep 0`;
- each retention group keeps exactly its `--keep` most recent installs
  that nothing else holds (ties broken by key) and removes the rest, and
  raising `--keep` never removes more;
- every install is kept or removed exactly once, and each kept install is
  reported with the first hold reason that applies;
- every selected key resolves, through the real `select_runtime_manifest`,
  to that same install rather than to a case twin of it.

The expected groups and holds are computed in the test from the `--keep`
help and the matching rules the policy implements, not with the policy's
own helpers. Generators draw from small, weighted alphabets so case-twin
keys, shared runtime ids, tied install times, flag/specs disagreement on
the toolchain choice and groups with more candidates than `--keep`
actually occur; a deterministic reach report fails if they stop.

A second property, 256 cases, applies whole plans to real registry files
and install folders through the real removal call and checks the install
in use survives.

The checked-in regression seed is a case-twin input that fails when the
runtime selector fix from #497 is reverted. docs/testing.md gains a
"Removing Old Installs" section listing the commands.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
rominf added a commit that referenced this pull request Nov 17, 2026
…ew points

Main gained an architecture-doc check that exempts property-test files only
by a `_properties`-style stem, so the bare `storage/properties.rs` would fail
it. Move it to `storage_properties.rs`, declared through `#[path]` as the same
`properties` module (as `process_tree_properties.rs` is), so test paths and
the documented `storage::properties` filter are unchanged, and give it an
architecture row like its sibling property files.

Drop the saved proptest seed instead of carrying it to the new name. It held
the #497 case-twin failure as an RNG seed, which replays a different input
once the generators change; that input is pinned as the literal example
`a_case_twin_pair_resolves_to_the_install_the_policy_cleared`, so the seed was
redundant, and left in place it would also be orphaned by the move.

Read both suites' case counts through `proptest_cases`, so `PROPTEST_CASES`
can raise them as it does for the other suites.

Count the four single-install toolchain rows of `generator_reach_report` in
groups of one too: a lone install whose toolchain is misread can still move
into a sibling group. Floors re-measured at four fifths of the new counts.

Correct what the resolver property is said to guard: generated case twins do
fail it when the resolver tries a case-insensitive match first (checked by
reverting that precedence), so it is not only the pinned example that covers
#497. State why the hold-reason oracle follows the policy's own order, and
say which tests the `storage::properties` filter runs besides properties.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtimes: prune de-registers the wrong install when two registry keys differ only in letter case

2 participants